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Abstract 

As post-school outcomes for di- 
verse students with disabilities 
continue to fall short of those of 
their majority -culture peers, there 
is an increasingly urgent need to 
increase multicultural compe- 
tence among transition profes- 
sionals. This study used a pre-/ 
post-test measure to assess 
changes in multicultural compe- 
tence for a statewide group of 
education and rehabilitation staff 
following a one-time-only train- 
ing. Findings produced mixed 
but limited results as to the effi- 
cacy of such training in 
multicultural competence. This 
suggests that short-term training 
may not produce powerful 
enough effects to impart long 
term changes; however, the re- 
sults also imply that merely sur- 
veying staff about attitudes and 
practices may act to raise aware- 
ness and increase competence. 
Implications for future research 
and practice are discussed. 


American public schools are 
serving an increasingly diverse 
student population. Growing 
ethnic and linguistic diversity in 
the United States has resulted 
in corresponding changes in the 
public schools (Luft, 2001; Sue, 
Bingham, Porche-Burke, & 
Vasquez, 1999). Some have es- 
timated that by the year 2040, 
culturally and linguistically di- 
verse (CLD) students will repre- 
sent more than one-half of the 
general school population (Leake 
& Black, 2005). 

Many of these CLD students 
and families will receive special 
education services. However, 
special education professionals 
may not be adequately prepared 
to provide effective career and 
transition services to meet the 
unique needs of these diverse 
students and their families. A 
number of recent studies have 
found that CLD families report 
a lack of participation in, and 
significant dissatisfaction with, 
special education and transition 
services (Defur, Todd-Allen, & 
Getzel, 2001; Geenen, Powers, 
Lopez-Vasquez, & Bersani, 
2003; Kim & Morningstar, 
2005). Greene and Nefsky (1999) 
summarized some of the major 
barriers that may deter active 
participation and equal owner- 
ship in transition planning for 
CLD families, including: (a) fam- 
ily knowledge of and comfort 
with school systems and proce- 
dures, (b) differences in interper- 
sonal communication styles or 
language barriers, (c) differing 
cultural attitudes towards dis- 
ability, and (d) lack of sensitiv- 
ity of special education staff to 
cultural diversity. In a recent 
literature review focused on CLD 
family involvement in transition 


planning, Kim and Morningstar 
(2005) found that professional 
attitudes, contextual barriers 
(such as poverty, time conflicts, 
and single parenthood), and 
bureaucratic barriers within the 
school and community all cre- 
ated significant obstacles to par- 
ticipation for diverse families. 
Given the importance of family 
participation in educational 
development (Newman, 2004), it 
is not surprising that outcomes 
for CLD students in special edu- 
cation lag significantly behind 
their majority-culture peers (Na- 
tional Center on Secondary Edu- 
cation and Transition, 2003; 
Trainor, Lindstrom, Simon- 
Burroughs, Sorrells, & Martin, 
in press; Wagner, Newman, 
Cameto, Garza & Levine, 2005). 

The need to address family 
participation and consistently 
poorer outcomes for CLD stu- 
dents has led to a call for greater 
multicultural competence 
among special education and 
transition personnel (Greene 
1996; Harry, 2002; Kim & 
Morningstar, 2005; Trainor, 
2005; Trainor et al., in press). 
Commonly, multicultural com- 
petence has been assumed to 
encompass three major catego- 
ries: awareness of diversity and 
of one’s own attitudes toward 
diversity; knowledge of minority 
populations and specific behav- 
ioral patterns/trends; and skills 
in communicating and working 
with a diverse range of people 
(Constantine, 2001; D’Andrea, 
Daniels, & Heck, 1991; Kerka, 
1992). Cultural competence in- 
cludes the ability to consider 
one’s own assumptions and be- 
liefs; evaluate the influence of 
culture on others; and respect 
differences that may be a result 
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of variations in family and cul- 
tural values (Greene, 1996; 
Harry, 2002; Luft, 2001). In 
part, cultural competence in- 
volves understanding how a 
youth’s challenging behavior 
may not necessarily represent 
“bad choices” or “faulty values”, 
but reasonable adaptations to 
cultural and contextual circum- 
stances (Geenen et. al, 2003; 
Hawks & Muha, 1991; Kalyanpur 
& Harry, 1997). 

Although the field of special 
education has only recently be- 
gun to recognize the importance 
of multicultural competence, 
other disciplines such as school 
counseling and health care edu- 
cation have highlighted the need 
for training in multicultural 
competence (e.g., Kai, Spencer, 
& Woodward, 2001; Carey, 
Reinat, & Fontes, 1990). In a 
large survey of school counse- 
lors, Carey, Reinat and Fontes 
(1990) reported that counselors 
believed that training was 
needed in the areas of improv- 
ing academic achievement of 
ethnic minority students, cross- 
cultural communication, and 
racism awareness. Researchers 
have also noted the benefits that 
increasing staff competence 
could have in offering culturally 
appropriate services for specific 
ethnic minority groups. In a 
study of career outcomes for 
Mexican-American high school 
girls, McWhirter, Hackett, and 
Bandalos (1998) found that 
teacher support acted to mod- 
erate the effect of perceived 
barriers, suggesting that educators 
well-versed in multicultural com- 
petence could make a significant 
impact on outcomes for students. 

The field of counseling psy- 
chology has been a primary 
leader in tackling the question 
of how multicultural competence 
might be improved in personnel. 
A recent meta-analysis reported 
positive associations between 
multicultural education and 
multicultural competence, par- 


ticularly in the areas of practi- 
tioners’ knowledge, awareness, 
and skills (Smith, Constantine, 
Dunn, Dinehart, & Montoya, 
2006). This study also found 
that interventions based on ex- 
tant theory have higher effect 
sizes than those not based on 
best practices literature (Smith 
et al., 2006). Similarly, experi- 
ential activities have shown an 
especially large benefit in skill 
development (Heppner & O’Brien, 
1994). Conversely, didactic or 
theory-oriented trainings have 
been reported to be less helpful 
or even detrimental to partici- 
pants (Kai, Spencer, & Wood- 
ward, 2001). These studies have 
largely been conducted with 
graduate students in counseling 
or other fields, and therefore the 
knowledge base on the effective- 
ness of diversity training for 
school based personnel, includ- 
ing special education and tran- 
sition professionals, remains 
limited at best. 

Purpose and Objectives 

The purpose of our study was to 
evaluate the need for, and effi- 
cacy of, a one-time-only diver- 
sity training for a group of edu- 
cation and rehabilitation profes- 
sionals who provide transition 
services for youth with disabili- 
ties. We were interested in col- 
lecting information on the issues 
and challenges transition staff 
may face in serving a more di- 
verse student population, and 
determining the value and im- 
pact of our training in 
multicultural competence. Such 
a short training may not be the 
“gold standard” for multicultural 
education; however, given the 
limited resources and time avail- 
able for training practicing pro- 
fessionals, this study addresses 
the question of whether such 
short and presumably common 
trainings provide any lasting 
benefit. In order to study these 
issues, we developed an instru- 
ment (the Diversity Self-Assess- 


ment) to measure self-reported 
multicultural competence, em- 
ployed a quasi-experimental 
control group, and followed up 
with participants two months 
after the initial intervention. We 
expected that our training would 
produce a small but measurable 
effect in the self-reported 
multicultural competence of 
those who attended. 

Method 

Context for the Training 

The diversity training evaluated 
in this study was one component 
of ongoing professional develop- 
ment activities provided to edu- 
cation and rehabilitation per- 
sonnel affiliated with the Youth 
Transition Program (YTP). The 
YTP is a partnership among the 
Oregon Office of Vocational Re- 
habilitation Services (OVRS), the 
Oregon Department of Educa- 
tion and local schools statewide 
in Oregon. The purpose of the 
program is to prepare high 
school youth with disabilities for 
employment- or career-related 
post-secondary education or 
training. YTP services include: 
transition planning and career 
exploration, school and commu- 
nity-based instruction, paid job 
training, and follow-up support 
services. Services are provided 
by transition specialists, who are 
primarily paraprofessionals em- 
ployed by the high schools in 
collaboration with counselors 
from OVRS. Currently, the YTP 
is being implemented in over 
120 high schools serving ap- 
proximately 1460 youth with 
disabilities. YTP is an evidence- 
based program which has been 
shown to significantly improve 
post-school outcomes for youth 
with disabilities (Benz, Lindstrom, 
& Latta, 1999; Benz, Lindstrom, 
& Yovanoff, 2000). Over the last 
five years, the program has 
served an increasing number of 
culturally and linguistically di- 
verse youth; in addition, more 
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than 50% of participants are 
from low income families 
(Lindstrom, Lichtenstein, & 
McGrath-Kato, 2006). 

Sample 

As our target population was 
personnel who could impact a 
Special Education student’s 
transition services, we collected 
data for this study from 63 edu- 
cation and rehabilitation profes- 
sionals who attended a full day 
statewide YTP training event (53 
staff members completed both 
pre- and post-test, and form the 
actual sample analyzed). The 
event served staff from across 
the state of Oregon and drew a 
wide variety of school and Vo- 
cational Rehabilitation staff. 
The training included sessions 
covering a variety of topics re- 
lated to improving transition 
services and outcomes for high 
school youth with disabilities. 
Of the 53 staff who completed 
both pre- and post-tests, 15 at- 
tended a one hour diversity ses- 
sion; 38 people did not attend 
and therefore formed our quasi- 
experimental control group. 
Study participants included 
school transition specialists 
(n=47), counselors from the Of- 
fice of Vocational Rehabilitation 
Services (OVRS) (n=8), and eight 
people representing other school 
staff, including special educa- 
tion directors or other adminis- 
trators. The sample included 
individuals from both rural and 
non-rural communities, and 
from a wide range of school 
sizes. The participants had 
spent an average of five years 
working in their particular roles, 
although experience ranged 
widely from 1 to 35 years. Of 
note, 84% of the sample identi- 
fied themselves as Caucasian/ 
European-American. While such 
limited diversity within the staff 
is undoubtedly an important 
challenge to address in our ef- 
forts to better serve diverse stu- 
dents, our sample appears to be 
relatively representative of the 


broader teaching force, which is 
largely female and European- 
American (National Center for 
Education Statistics, 2005; Na- 
tional Collaborative on Diversity 
in the Teaching Force, 2004). No 
significant demographic differ- 
ences (e.g., gender, ethnicity, or 
years in role) were present be- 
tween the “trained” and “un- 
trained” groups. (See complete 
participant demographics in 
Table 1.) 

Description of the Diversity 
Training 

Our diversity training was of- 
fered as an optional breakout 
session during an annual state- 
wide YTP conference. Training 
content was based on the expe- 
rience of the authors in special 
education and counseling, for- 
mal education in multicultural 
competence, and a thorough re- 
view of the literature in the area 
of diversity and transition. 
Therefore, the training was con- 
sidered to represent “best prac- 
tices” in multicultural compe- 
tence and transition (Kai, Spen- 
cer & Woodward, 200 1 ; Leake & 
Cholmay, 2004; Smith et al. 
2006). The training was com- 
prised of several components, 
including: information on post- 
school outcomes for ethnic mi- 


nority and low-income youth, 
interactive discussions of case 
studies, review of strategies and 
resources for improving service 
to diverse students, and encour- 
agement to set “action plans” to 
determine how the information 
could be translated into prac- 
tice. Where possible, the evi- 
dence base was incorporated 
into both content (e.g., critical 
principles to stress in working 
with diverse families) and meth- 
odology (e.g., interactive, expe- 
riential activities) of the train- 
ing. For more information on the 
content and format of the train- 
ing please contact the first au- 
thor. The diversity training was 
repeated at two sessions during 
the conference in order to maxi- 
mize attendance. 

Data Collection and 
Measurement 

Data Collection Procedures 
We collected data for this study 
in two waves. First, we distrib- 
uted the Diversity Self-Assess- 
ment instrument (described in 
detail below) to all participants 
attending the statewide YTP con- 
ference. The instrument was 
administered during a large 
group session prior to the diver- 
sity training to maximize partici- 
pation. In total, 63 YTP staff 


Table 1 

Participant Demographic Data (N = 

63) 


Variable 


N 

% 

Gender 

Female 

56 

90 


Male 

6 

10 


Missing Data 

1 


Ethnicity 

Caucasian 

53 

84 


Ethnic Minority 

5 

8 


Missing data 

5 

8 

Role 

Transition Specialist 

46 

74 


VR Counselor 

8 

13 


Other School Staff 

8 

13 

Years in Position 

1-3 years 

33 

53 


4 - 7 years 

14 

23 


8-10 years 

8 

13 


11-35 

7 

11 

School Location 

Nonrural 

34 

55 


Rural 

28 

45 

Attended Diversity Training 

Did not attend 

45 

71 


Attended 

18 

29 
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members completed the pre-test 
self-assessment. Approximately 
two months later, we contacted 
all participants who completed 
the pre-test to inform them of 
the need for follow-up. Then, 
over the course of two weeks, we 
completed post-test telephone 
interviews using the same in- 
strument (with slight modifica- 
tions to the open-ended ques- 
tions) with 53 of the original 
survey respondents. We made 
multiple efforts to retain partici- 
pants, ultimately achieving an 
84% retention rate. 

Instrument development 
The Diversity Self-Assessment is 
a 22-item instrument that asks 
participants to rate themselves 
on a range of questions related 
to awareness, knowledge, and 
skills for serving diverse popu- 
lations of students with disabili- 
ties. Participants rated them- 
selves on a four-point Likert- 
type scale that ranged from 1 
[rarely/ strongly disagree) to 4 
(always/ strongly agree). In ad- 
dition, the pre-test assessment 
asked two open-ended questions 
about challenges and needs 
identified by staff in working 
with diverse populations. We 
included three additional open- 
ended questions about changes 
in service delivery in the post- 
test assessment. Six items ques- 
tioning participants about the 
population they serve and the 
technical assistance they re- 
ceive, were created for internal 
purposes and were not included 
in these analyses. 

The Diversity Self-Assess- 
ment was created following an 
extensive review of the literature 
on diversity in special education 
and transition. Items were cre- 
ated that reflected important 
concepts from the literature 
(e.g., flexibility to family needs) 
or significant barriers identified 
(e.g., language barriers). The 
original version of the instru- 
ment was pilot-tested at an ear- 
lier training session and ex- 


panded based on feedback from 
that session. Our instrument 
was developed to assess three 
major concepts: (a) awareness of 
the influence of culture; (b) 
knowledge of specific cultural 
practices and values of diverse 
families; and (c) skills in work- 
ing with a diverse range of stu- 
dents and families. This empha- 
sis on awareness, knowledge, 
and skills is supported by exist- 
ing literature on assessing 
multicultural competence 
(D’Andrea, Daniels, & Heck, 
1991). Given the well-docu- 
mented importance of working 
with families (Geenen, Powers, 
& Lopez-Vasquez, 2001; 
Kalyanpur & Harry, 1997; Kim 
& Morningstar, 2005) and devel- 
oping self-awareness (Harry, 
2002; Kerka, 1992; Stuart, 
2004), many of our questions 
highlighted these areas within 
the three essential qualities of 
awareness, knowledge, or skills. 
Others have followed a similar 
process of instrument develop- 
ment (e.g., Carey, Reinat, & 
Fontes, 1990) to measure this 
construct. Although it carries 
significant limitations, self-re- 
port measurement is the most 
common approach to assessing 
multicultural competence 
(Smith et al., 2006). 

We designed the Diversity 
Self-Assessment instrument to 
target three broad subcategories 
of multicultural competence 
(e.g., awareness, knowledge, and 
skills); however, we also sought 
an empirical approach to creat- 
ing subscales to validate our ini- 
tial hypotheses. Therefore, we 
employed Principal Components 
Analysis (PCA) with Varimax ro- 
tation to determine empirical 
scales based on inter-item cor- 
relation and underlying factor 
analysis. PCA has been reported 
to be an appropriate means of 
initial data-reduction to produce 
components useful for future 
analyses (Tabachnick & Fidell, 
2001). As a further check on the 


methodology, exploratory factor 
analysis was also performed on 
the same variables and achieved 
essentially the same results as 
reported below. 

The Principal Components 
Analysis revealed that a four- 
factor solution best fit the data, 
explaining 61% of the variance 
in items while retaining ad- 
equate numbers of items for 
each factor. Solutions with more 
than four factors did not signifi- 
cantly improve the amount of 
variance explained. Although 
the four-factor solution did not 
exactly mirror our expectations, 
it did produce scales consistent 
with our understanding of the 
literature and important prin- 
ciples relevant to multicultural 
competence. The four factors 
included: (a) “Valuing Culture” 
(adequate reliability, a=.78), (b) 
“Cultural Awareness” (adequate 
reliability, a=.79); (c) “Family 
Involvement” (fair reliability, 
a=.58); and (d) “Self-awareness” 
(fair reliability, a=.58). See Table 
2 for a list of individual items 
that comprise each factor. This 
PCA covered all of the targeted 
items save two, which were not 
adequately differentiated be- 
tween the four factors and were 
accordingly excluded from fur- 
ther analysis. 

Data Analysis 

Data analysis proceeded in two 
major modes. First, we used 
ANCOVA to test for group differ- 
ences using scales derived from 
the Principle Components 
Analysis. Second, we used quali- 
tative methods such as thematic 
analysis to understand overall 
changes and group differences 
in the open-ended questions 
(Patton, 2002). 

Quantitative analysis 
First, we were interested in 
whether or not there were sig- 
nificant differences between the 
pre- and post-test scores for 
each scale across the whole 
sample. To address this question 
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Table 2 

Principal Components Analysis Four-Factor 
Solutions with Items 

Factor 

Items 

Item Factor 
Loading 

“Valuing 

13.1 encourage students to explore and 

.782 

Culture” 

discuss their heritage, cultural back- 
ground, and/or family values, and 
to set goals that are consistent with 
this background. 



15. In meetings with families and/or 
youth I discuss the young person’s 
strengths and abilities. 

.623 


16. I seek information about people of 
diverse backgrounds, cultures and 
experiences. 

.641 


18. I search for strengths related to 
the youth’s family and culture. 

.757 


19. When faced with a challenging or 
difficult case, I consider the influence 
of family or cultural values. 

.697 

“Cultural 

2. I acknowledge many different kinds 

.749 

Aware - 

of families and feel comfortable work- 


ness” 

ing with them in the transition pro- 
cess. 



4. I am familiar with local or state 
trends regarding changes in popu- 
lation and outcomes for diverse 
groups. 

.817 


6. I am aware of services, organiza- 
tions, or locations that are frequently 
accessed or visited by members of 
ethnic/ racial minority, low income, 
or other groups. 

.738 


7. 1 am familiar with cultural practices, 
norms, or values of a variety of eth- 
nic, religious, or other groups. 

.748 

“Family 

8. I explain to the youth /families that 

.738 

Involve- 

I work with the transition process, 


ment” 

options for services, and ways in 
which they can actively participate 
in the process. 



10. I actively encourage and seek par- 
ticipation from families. 

.648 


12. I am flexible with meeting times 
and locations in order to fully include 
vouth and their families. 

.764 

“Self- 

3. I am aware of how my own cultural 

.783 

Aware - 

background/ values may influence 


ness” 

the goals I encourage youth to set or 
my expectations for post school out- 
comes. 



2 1 . When faced with a challenging or 
difficult case, I consider the influence 
of mv own familv or cultural values. 

.846 


we performed paired-samples t- 
tests for overall differences be- 
tween pre- and post-test scores 
on each scale. We expected that, 
in the absence of training, we 
would not observe overall differ- 
ences for the whole sample be- 
tween pre- and post-test. Next, 
we tested the effect of the diver- 
sity training on the Diversity 
Self-Assessment scores. To do 
this, we ran between-groups 
analysis of covariance (ANCOVA), 
searching for differences between 
the trained and untrained 
groups in the post-test score, 
holding the pre-test score as a 
covariate. We expected to see 
small but significant increases 
on the post-test for the trained 
group, relative to the untrained 
group. Finally, to examine the 
influence of demographic factors 
on our measure, we searched for 
significant correlations between 
both pre- and post-test scores 
and gender, ethnicity, years in 
role, and rural/ non-rural status. 

Qualitative analysis 
We analyzed the responses to 
the open-ended questions using 
standard techniques for quali- 
tative analysis (Miles & 
Huberman, 1994; Patton, 2002). 
First, participant responses 
were typed (or transcribed in the 
case of the phone interviews) 
into a single document. Two of 
the co-authors reviewed all of 
the open-ended responses and 
developed a set of codes based 
on common responses. Next all 
of the open-ended responses 
were coded and summarized in 
tables, placing responses in 
rank order using the codes as 
an organizing structure. 

Results 

Quantitative Analyses 

Pre-test 

We examined the pre-test scores 
for each scale to understand 
how staff perceived their 
multicultural competence in the 
absence of any formal training. 
At pre-test, the mean overall re- 
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sponse was 3. 1 out of a possible 
4.0, indicating that staff, on av- 
erage, felt very positively about 
their frequency of use of the 
strategies measured in the in- 
strument. (See Table 3.) For in- 
stance, participants felt that 
they commonly acknowledged 
various family structures, flexed 
their style to meet the varying 
needs of families, and/or fo- 
cused on a youth’s strengths 
and abilities. For each of the four 
scales derived, staff rated them- 
selves on average at a three or 


above, suggesting positive esti- 
mates of their skills and aware- 
ness of diverse populations (and 
perhaps making it more difficult 
for significant intervention ef- 
fects to emerge.) Interestingly, 
those who attended the diversity 
training rated themselves lower 
on all but one of the pre-test 
scales (and all but two of the 
post-test scales). 

Post-test 

To assess whether there was a 
change in multicultural compe- 
tence as a result of the diversity 


trainings, we first analyzed 
whether, for the whole group, 
there was a net increase in 
scores from pre- to post-test. 
Contrary to our expectations, 
paired samples t-tests indicated 
that for most scales, including 
the total score, there were small 
but significant gains in mean 
scores from pre to post-test for 
the whole sample (combining 
trained and untrained groups). 
(See Table 3.) The only scale that 
did not show significant change 
was the Cultural Awareness scale. 


Table 3 

Overall Differences for Each Scale (Pre and Post-test) 

Scale 

Pre-test 

Mean 

(item)* 

Post-test 

Mean 

(item)* 

Mean 

difference 

(scale)** 

t 

df 

Significance 

Total score 

3.14 

3.33 

3.15 

2.62 

52 

.01 

Valuing Culture 

3.13 

3.34 

1.24 

3.78 

51 

<.001 

Family Involvement 

3.42 

3.69 

.83 

4.30 

51 

<.001 

Self Awareness 

3.03 

3.24 

.58 

2.73 

52 

<.01 

Cultural Awareness 

3.02 

3.11 

.26 

1.02 

52 

.31 

Note: This table reflects the sample for whom both pre- and post-test data were available. 

*Item means reflect the mean score for items within a given : 

scale 



** Scale mean differences reflect the mean differences for the summed items in 

a scale between pre 

and post-test 








Table 4 

Group Comparisons (Trained versus Untrained) for Post-test Scores 


Scale 

Pretest Mean 
(Standard Deviation) 

Post -test Mean 
(Standard Deviation) 

Valuing Culture 

Untrained 

15.5 (2.9) 

16.4 (2.4)* 

Trained 

16.0 (2.9) 

17.3 (1.5) 

Cultural Awareness 

Untrained 

12.1 (2.3) 

12.6 (1.9) 

Trained 

11.7 (2.0) 

11.5 (1.4) 

Family Involvement 

Untrained 

10.5 (1.2) 

11.2 (1.0) 

Trained 

9.7 (2.2) 

10.8 (1.0) 

Self-Awareness 

Untrained 

5.9 (1.8) 

6.4 (1.2) 

Trained 

5.8 (1.3) 

6.6 (1.2) 

Overall Score 

Untrained 

51.0 (9.8) 

53.1 (5.8) 

Trained 

45.9 (11.5) 

52.6 (4.0) 

* Significant ANCOVA (p< 

.05) (significant increase for untrained group relative to trained) 
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Next, we compared the rela- 
tive change in pre- to post-test 
scores for each scale by group 
(trained versus untrained). Re- 
sults of this analysis appear in 
Table 4. The data suggest that, 
overall, there was no apparent 
improvement in self-reported 
multicultural competence attrib- 
utable to the diversity training. 
In fact, of the five ANCOVA 
analyses conducted (one for 
each scale and the total score), 
only one was significant, and 
that was for the “Valuing Cul- 
ture” scale, in which the un- 
trained group’s scores increased 
while the trained group’s scores 
decreased. Otherwise, there 
were no significant ANCOVA 
scores, suggesting that when 
controlling for pre-test scores, 


there were no differences be- 
tween trained and untrained 
groups on post-test competence 
scores. Furthermore, staff demo- 
graphic variables were not re- 
lated to their levels of 
multicultural competence. We 
examined correlations between 
role, years in role, gender, and 
ethnicity and all of the pre-/ 
post-test scores; there were no 
significant correlations beyond 
what would be expected from 
random error. 

Qualitative Analyses 

Pre-test findings 

The open-ended questions in our 
Diversity Self-Assessment in- 
strument allowed us to conduct 
a more in-depth examination of 
the perceptions of transition 


staff during the pre- and post- 
test administration. We exam- 
ined the pre-test data to gain an 
initial understanding of the 
types of information transition 
staff needed to be able to better 
serve diverse students. Table 5 
provides a summary of the most 
common responses to our open- 
ended question regarding infor- 
mation needs. Responses are 
summarized by themes and 
listed in rank order. The high- 
est priority need identified by 
education and rehabilitation 
professionals was for basic in- 
formation regarding diverse 
families, including a more com- 
prehensive framework for un- 
derstanding cultural norms, val- 
ues, and expectations. One as- 
pect of this theme was a request 


Table 5 

Information Needs Identified by Participants at Pre-test 


Rank 

Order Key Theme Issues/Topics Identified by Participants 


1 


General Cultural In- 
formation 


•Understanding cultural norms and values 

•Awareness of family expectations and attitudes toward education, 
employment, and post school services 
•Information about population changes; demographic trends 


2 Strategies for High Providing services for diverse youth who may face additional barri- 

Risk Youth ers including: 

•Homelessness 

•Poverty 

•Drug involvement 
•Mental health concerns 
•Adjudicated youth 

3 Language Information ‘Classes to improve language skills (primarily Spanish) 

•Translation services for program materials 
•Job placement strategies for bilingual students 


4 


5 

6 


Working with Immi- 
grant Populations 


•Process for working with youth & families who are not US citizens 
•College options and planning 
•Obtaining green cards or work permits 


Community Supports ’Information on community programs and services for diverse youth 

•Access to community mentors and advocates 


Policy/ System Level 
Support 


•Encouragement at the state level to serve diverse youth in transi- 
tion programs 
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for additional information on 
demographic trends and the in- 
creasingly diverse student popu- 
lation served within their pro- 
grams. The second most com- 
mon need identified was in the 
area of strategies and resources 
for diverse youth facing barriers 
such as homelessness, poverty, 
or drug involvement. Partici- 
pants were also interested in 
having access to language ser- 
vices and specific information to 
work with youth from immigrant 
families. Finally, participants 
identified a need for information 
about local community support 
systems and a desire for state 
level support to improve transi- 
tion services and outcomes for 
diverse youth. 

Post-test findings 
During the post-test interviews, 
we asked participants if they 
had made any changes in the 
way they provide career or tran- 
sition services as a result of at- 
tending the training or complet- 
ing the Diversity Self-Assess- 
ment. (See Table 6.) Consistent 
with our quantitative analyses, 
we found that some of the tran- 
sition personnel in both the 
trained and untrained groups 
reported changing their prac- 


tices or expanding their aware- 
ness. However, individuals in 
the trained group were more 
likely to indicate specific 
changes in planning, referral, or 
other service delivery strategies 
(60% versus 26%). Consistent 
with this finding, those partici- 
pants who completed the initial 
Diversity Self-Assessment, but 
did not attend the diversity 
training, more often indicated no 
changes in service delivery (48% 
versus 20%). Conversely, they 
were somewhat more likely to 
indicate an increase in overall di- 
versity awareness (26 % vs. 20%). 

Participants who described 
making changes in their 
multicultural awareness or 
practice were asked to provide 
specific examples of the changes 
they had made. A summary of 
these responses (listed in rank 
order) is included in Table 7. The 
most common change described 
was for staff to actively seek in- 
formation about families’ cul- 
tural context, and to incorporate 
these cultural values and expec- 
tations into the transition pro- 
cess. Many participants also 
described an increase in their 
own awareness of diversity is- 
sues and a broader understand- 
ing of the importance of cultural 


context. In addition, staff noted that 
they were more likely to refer stu- 
dents to community resources and 
services and to have program in- 
formation or forms translated into 
a variety of languages. 

Discussion 

This study provides several con- 
tributions to the existing tran- 
sition and multicultural training 
literature. Overall, we found 
fairly high initial levels of self- 
reported multicultural compe- 
tence among the education and 
rehabilitation professionals par- 
ticipating in the study. During 
the pre-test, staff also identified 
a number of specific training 
needs to increase their knowl- 
edge and skills in working with 
diverse populations. Analysis of 
post-test scores resulted in 
small but significant gains for all 
staff, regardless of participation 
in the diversity training session. 
Although we found only one sig- 
nificant difference between the 
trained and untrained groups, 
responses to the open-ended 
post-test questions indicated a 
number of changes in service 
delivery based on participation 
in the assessment or diversity 
training session. Finally, there 
appeared to be no significant 


Table 6 

Post-test Findings: Participant Changes across Trained and Untrained Groups (N = 53) 



T r a 
(N = 

i n e d 
= 15) 

Untrained 
(N = 38) 

Sample Participant Responses 


N 

% 

N 

% 


Yes: Made 
Changes in 
Services 

9 

60 

10 

26 

I now consciously consider family background, 
heritage, and what the student brings to the 
table. 

I have a greater understanding of the many di- 
versity issues at play. 

No Changes 

3 

20 

18 

48 

No. currently there is not much cultural or eth- 
nic diversity in our district, but I see this chang- 
ing in the next few years. 

Increased 

Awareness 

3 

20 

10 

26 

Not really, but I have increased my awarenes and 
continue to grow in this area. 
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relationship between demo- 
graphic characteristics of the 
staff and multicultural compe- 
tence outcomes. Below we con- 
sider these findings in depth. 

The Diversity Self-Assessment 

Based on our Principal Compo- 
nents Analysis, the Diversity 
Self-Assessment was able to 
generate four empirically derived 
scales of fair to adequate reliabil- 
ity. Given the small number of 
items, this appears to be a suc- 
cessful initial effort to reliably 
measure multicultural compe- 
tence for transition staff. The 
question of validity, limited by 
the self-report nature of the in- 
strument and the lack of other 
instruments to use as compari- 
son, is more difficult to answer. 
In the absence of more formal 
proof, we can offer circumstan- 
tial evidence of the validity of 
this instrument. First, it is 
firmly based in the empirical lit- 
erature of both multiculturalism 
and transition, thus making it 
more likely to accurately repre- 
sent the overlapping demands of 
those content areas. Second, the 
instrument was pilot-tested with 
an earlier group of transition 
staff and revised based on those 
pilot tests. Third, feedback from 


the staff in open-ended ques- 
tions allowed us an alternative 
way of examining the evidence 
for any finding. Therefore, we 
believe that this instrument is, 
overall, a reasonably valid tool 
in measuring multicultural com- 
petence for transition staff. The 
Diversity Self- Assessment may 
serve as a critical first step for 
program developers and evalu- 
ators to understand the need for 
and efficacy of training in diver- 
sity. Interestingly, although the 
scales did not conform to our 
original expectations of aware- 
ness, knowledge, and skills, they 
cohere around factors that reso- 
nate with scholarly estimates of 
what composes multicultural 
competence (Harry, 2002; 
Kerka, 1992; Stuart, 2004). That 
is, qualities such as valuing cul- 
tural diversity, seeking to in- 
clude families, and developing 
self-awareness seem well-suited 
to an assessment of a transition 
staff member’s multicultural 
competence. 

Initial Levels of Multicultural 
Competence 

By and large, staff initially rated 
themselves as having fairly high 
degrees of multicultural compe- 
tence. During the pre-test as- 


sessment, the mean value for 
most scales was at or above a 
three out of four on a Likert-type 
scale. Staff felt that they com- 
monly considered cultural vari- 
ables, were flexible with their 
meeting styles and their ap- 
proaches to families, and fo- 
cused on strengths. Conversely, 
on the open-ended qualitative 
questions, participants sug- 
gested a number of areas in 
which they would like more in- 
formation or training related to 
working with diverse popula- 
tions. Taken together, these re- 
sults present a mixed picture of 
the participants’ initial level of 
multicultural competence. On 
the one hand, they suggest that 
staff surveyed may not need 
training in multicultural compe- 
tence - after all, based on their 
quantitative data they seemed to 
feel quite comfortable thinking 
about and embodying conscien- 
tious attitudes about culturally 
diverse students. On the other 
hand, the open-ended responses 
suggested numerous areas in 
which staff reported that they 
would benefit from training. 
Given this, and given a litera- 
ture that suggests that diverse 
families are frequently 
underserved or inadequately 



Table 7 

Changes Identified in Transition Service Delivery 

Rank 



Order 

Key Theme 

Changes Identified by Participants 

1 

Seek More Information from/ 
about Families 

•Actively seek input from families in career and transition 
planning process 

•Consider family background/ culture in decision making 

2 

Connect Students to Specific 
Resources 

•Accessing specific programs to meet needs of diverse stu- 
dents (e.g. multicultural center at local college) 
•Building connections with local employers and trade or- 
ganizations 

3 

Increased Awareness of Di- 
versity Issues 

•Broader view of cultural and family values 
•Understanding of “my own assumptions” 

4 

Sought Information in Other 
Formats 

•Requested translation of forms and materials 
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served by transition staff (Defur, 
Todd-Allen, & Getzel, 2001; 
Greene & Nefsky, 1999; Trainor, 
et al., in press), it is possible that 
the participants’ self-ratings 
were artificially inflated. That is, 
transition staff may have over- 
estimated the frequency and 
consistency with which they 
embody the skills and values 
assessed in the survey, and thus 
underestimated their need for 
further training. 

Possible Survey Effects 

We had not intended the Diver- 
sity Self-Assessment to promote 
any changes in attitudes or 
practice. However, the data from 
both quantitative and qualitative 
measures suggest that there was 
a small but significant increase 
from pre- to post-test for the 
group as a whole. In studies over 
longer spans of time, such an 
increase might suggest a devel- 
opmental change in which, over 
time, an entire group of people 
improves in their abilities in a 
given area. However, given the 
limited time difference (two 
months) between pre- and post- 
testing and the fact that this 
survey was conducted with ma- 
ture, experienced staff in the 
field, this seems unlikely. What 
seems more likely is that there 
was a small intervention effect 
of surveying staff about their 
multicultural competence. In 
other words, by asking ques- 
tions of everyone about their in- 
teractions with diverse students 
and families, we may have en- 
couraged further thought and 
growth around these issues. 
Thus, our Diversity Self-Assess- 
ment may have served to not 
only gather information about 
staff multicultural competence, 
but to actually improve that 
competence by raising dialogue 
and awareness. 

Effects of Diversity Training 

As mentioned above, effects of 
diversity training may be par- 
ticularly difficult to assess in 


this study. The self-report mea- 
sure necessarily relies on a staff 
person’s own views of their abili- 
ties and may not reflect subtle 
changes inculcated by a con- 
sciousness-raising effort. And 
given our emphasis on aware- 
ness, staff may have come to 
realize how much more there is 
to learn about working with di- 
verse populations, and thus 
rated themselves lower on the 
self-report instrument following 
training. In fact, the staff who 
attended the diversity training 
did indeed rate themselves lower 
on one scale following the diver- 
sity training than the untrained 
group. Finally, most staff had 
already rated themselves fairly 
highly on the pre-test measure 
and therefore a “ceiling effect” 
may have made differences even 
more difficult to detect. 

Given all this, our best esti- 
mate is that, although there was 
not a detectable impact of our 
training on the attendees’ quan- 
titative ratings, there may yet 
have been a small beneficial ef- 
fect of the training. The rating 
scales are fairly clear - in no case 
was there a significant improve- 
ment, relative to the untrained 
group, for the trained group. 
However, the open-ended ques- 
tions tell a different story. 
Whereas more than three-quar- 
ters of the staff who participated 
in the training could identify 
specific changes in their service 
provision or conceptualization, 
only one-third of those who did 
not participate in the training 
identified specific changes or 
improvements in service. Many 
more trained staff indicated that 
they used a multicultural frame- 
work to provide transition plan- 
ning and other services. This, 
then, suggests that, although 
there may have been changes in 
the practice of trained transition 
staff, they may not have been 
broad or powerful enough to be 
captured by the self-report rat- 
ings scales. 


Limitations of the Study 

One important point to note in 
considering the implications of 
these findings is that our study 
assumes that multicultural 
competence is a quality that can 
be measured in a written instru- 
ment, and discretely altered by 
brief training. In reality, 
multicultural competence is a 
complex construct, encompass- 
ing a range of attitudes, knowl- 
edge, and interpersonal qualities 
(Stuart, 2004; Smith et al., 
2006). Assessing multicultural 
competence may not easily lend 
itself to the kinds of methodolo- 
gies that have been used in stud- 
ies of other interventions. None- 
theless, we hope that studies 
such as ours will continue to 
encourage the discipline to de- 
velop both theoretically and 
practically in order to meet the 
important needs identified for 
CLD students and families in 
transition. 

Several of the limitations of 
this study stem from its meth- 
odology, which appears appro- 
priate given that this is an early 
examination in an emerging dis- 
cipline. To begin with, our small 
sample size limited our power to 
detect distinctions between 
groups. Our conclusions are also 
limited by the nature of our 
sample, in that the two groups 
were not randomly assigned, but 
rather self-selected whether or 
not they would attend the train- 
ing session. Although quite com- 
mon for this purpose, the self- 
report nature of our instrument 
may also be a limitation. Despite 
some correlational and anec- 
dotal findings, the literature re- 
ports little match between self- 
and other-reported multicultural 
competence (Constantine, 2001). 
Thus, self-report measures such 
as ours may instead assess 
“multicultural counseling self- 
efficacy,” which might further 
explain our contradictory find- 
ings from the ratings and open- 
ended questions. Despite these 
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limitations, we believe our study 
has value for its attempt to ad- 
dress a critical need with direct ser- 
vice transition staff, and for its pre- 
/post-test measurement of both 
quantitative and qualitative ratings 
of multicultural competence. 

Implications for Practice 

Although it has become clear 
that there is a need to improve 
transition-related service deliv- 
ery for diverse students and 
families, special educators have 
only recently begun to examine 
how to actually address such 
issues in practice. Our study 
suggests that short, one-time- 
only training may not be enough 
to induce consistent, wholesale 
changes in the way transition 
staff conceptualize and serve 
diverse populations. Training 
such as ours may increase staff 
awareness of diversity issues, or, 
as we discovered, spark some 
discreet changes in staff’s pat- 
tern of service delivery. There- 
fore we believe that a one-time 
training may have some benefi- 
cial impact. However, our data 
suggest that in order to truly 
transform the beliefs and prac- 
tices of staff providing transition 
services, longer training with 
more in-depth interactions and 
experiences may be required. 
Furthermore, follow-up sessions 
to extend impact and to reinforce 
the value of these strategies over 
the long term might prove useful. 

It may also be fruitful to find 
other ways to increase profes- 
sionals’ awareness of the limits 
of existing transition services for 
diverse families. This increased 
awareness may encourage staff 
to consider changes in their own 
practice by attending voluntary 
trainings and seeking additional 
information (only about one- 
third of our sample elected to 
attend our training) . This could 
be accomplished by document- 
ing poor post school outcomes 
for diverse students, highlight- 
ing how many staff find gaps in 
their knowledge regarding di- 


verse students and families, or 
other means. The implication 
here is that it may be useful to 
find ways to encourage staff to 
seek training - to create a sort of 
“readiness for change” interven- 
tion - prior to offering training 
sessions in multicultural practice. 

Implications for Research 

While providing hints at prom- 
ising directions for future train- 
ing and practice, this study 
raises more questions than it 
answers. One critical question 
raised is about the validity of 
measurements of multicultural 
competence. The literature 
notes, not surprisingly, that self- 
report measures are quite com- 
monly used in this arena, but 
have proven limited in illuminat- 
ing covert, subtle, or more glo- 
bal changes in multicultural 
competence (Smith et al., 2006). 
Nor is there a well-established 
correlation between self- and 
other-reported multicultural 
competence (Constantine, 
2001). But the value of self-re- 
port measures, both in terms of 
expediency and in terms of the 
possible intervention effect they 
might produce, suggest a great 
need for more research on the 
validity of self-report measures. 
Could such a measure ad- 
equately capture people’s actual 
practice? We suggest that open- 
ended or qualitative items pro- 
vide a valuable addition by gen- 
erating triangulating evidence as 
a validity check. In addition, 
given that the ultimate goal of 
this training was not to improve 
staff practice, but to create bet- 
ter outcomes for students, more 
information is needed on how 
well self-report measures for 
transition professionals might 
predict outcomes for diverse stu- 
dents. To date, there has been 
almost no investigation of how 
multicultural training impacts 
outcomes for clients (Smith et 
al., 2006). We believe that the 
Diversity Self-Assessment could 
serve as an important first step 


in the effort to quickly measure 
and impact multicultural compe- 
tence for professionals in the field. 

The fields of transition and 
multicultural education also 
need an increased emphasis on 
developing and evaluating 
multicultural training practices. 
Much of the literature in 
multicultural education comes 
in studying graduate students; 
further investigation is needed 
on multicultural competence for 
practicing professionals. As 
practitioners and educators, we 
need to understand whether cer- 
tain training strategies and tech- 
niques might be more effective, 
leading to improved multicultural 
competence in staff working on 
the front lines of education and 
mental health. 

Conclusion 

Post-school outcomes for diverse 
students with disabilities have 
continued to lag behind those of 
their majority-culture peers 
(Wagner et al., 2005). Increas- 
ing the numbers of well-trained, 
culturally competent service 
providers would make a signifi- 
cant impact on this imbalance. 
Although our study suggests 
that one-time-only training is 
probably insufficient to create 
sustained changes, we have dis- 
covered important implications 
for future work in this arena. 
Our experience with our own 
training efforts implies that pe- 
riodic assessment of staff mem- 
bers’ awareness and practices 
regarding diversity, coupled with 
in-depth, experiential, ongoing 
training, may very well improve 
the multicultural competence of 
transition staff. Broadly speak- 
ing, we believe our study sup- 
ports the notion that continu- 
ously examining the influence of 
culture and context will improve 
transition service delivery, and 
ultimately, begin to better ad- 
dress the multiple barriers that 
diverse youth with disabilities 
experience. 
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